Papers with supervised training

48 papers
TextGraphs 2019 Shared Task on Multi-Hop Inference for Explanation Regeneration (D19-53)

Copied to clipboard

Challenge: Detailed extended analyses of all submitted systems showed large relative improvements in accessing the most challenging multi-hop inference problems, while absolute performance remains low.
Approach: The Shared Task on Multi-Hop Inference for Explanation Regeneration asks participants to regenerate detailed gold explanations for elementary science questions by selecting facts from a knowledge base of semi-structured tables.
Outcome: The top-performing system achieved a mean average precision of 0.56 . the task combines facts from a knowledge base and supervised training data .
Towards Making the Most of Cross-Lingual Transfer for Zero-Shot Neural Machine Translation (2022.acl-long)

Copied to clipboard

Challenge: Existing unsupervised neural machine translation systems can degrade when labeled data is limited.
Approach: They propose a multilingual pretraining and multilingual fine-tuning for facilitating cross-lingual transfer in zero-shot translation using a parallel dataset.
Outcome: The proposed model outperforms state-of-the-art models on many-to-English translation by over 7.2 and 5.0 BLEU.
Exploiting Cloze-Questions for Few-Shot Text Classification and Natural Language Inference (2021.eacl-main)

Copied to clipboard

Challenge: Existing approaches to learning from examples are limited due to the vast number of languages, domains and tasks.
Approach: They propose a semi-supervised training procedure that reformulates input examples as cloze-style phrases to help language models understand a given task.
Outcome: The proposed approach outperforms supervised training and strong semi-supervised approaches in low-resource settings by a large margin.
Parameter Space Factorization for Zero-Shot Learning across Tasks and Languages (2021.tacl-1)

Copied to clipboard

Challenge: Currently, there are only 24 languages in the world that have not been annotated . transferring knowledge across domains is a common solution .
Approach: They propose a Bayesian generative model for the space of neural parameters that factorizes into latent variables for each language and each task.
Outcome: The proposed model can perform better than state-of-the-art methods with a typologically diverse sample of 33 languages from 4 continents and 11 families.
Guiding Neural Entity Alignment with Compatibility (2022.emnlp-main)

Copied to clipboard

Challenge: Entity Alignment (EA) aims to find equivalent entities between two Knowledge Graphs (KGs) labelled data is used to learn neural EA models, but this aspect is neglected .
Approach: They propose a framework to integrate compatibility into neural EA models . they aim to find equivalent entities between two Knowledge Graphs (KGs)
Outcome: The proposed framework can achieve comparable effectiveness with supervised training using 20% of labelled data.
Temporal Knowledge Graph Forecasting Without Knowledge Using In-Context Learning (2023.emnlp-main)

Copied to clipboard

Challenge: Temporal knowledge graphs (TKGs) are used to represent real-world facts in a structured way.
Approach: They propose to use in-context learning with large language models for TKG forecasting . they compare naive LLMs to state-of-the-art (SOTA) supervised models .
Outcome: The proposed approach performs well against pre-trained large language models . the proposed approach is based on simple heuristics and state-of-the-art models compared with pre-trainers .
Hierarchical Attention Prototypical Networks for Few-Shot Text Classification (D19-1)

Copied to clipboard

Challenge: Existing methods for text classification are based on large-scale labeled data, but few data are available.
Approach: They propose a hierarchical attention prototypical networks for few-shot text classification . they use attention mechanism to highlight or weaken the importance of features, words, and instances .
Outcome: The proposed model can capture more important features, words, and instances . it can also increase support set augmentability and accelerate convergence speed in training stage .
Regression-Free Model Updates for Spoken Language Understanding (2023.acl-industry)

Copied to clipboard

Challenge: Recent work has proposed methods for minimizing regressions caused by model updates . focus is on spoken language understanding models, which are unexplored .
Approach: They propose a focal distillation technique to reduce regressions in goal-oriented dialog systems . they also evaluate its effectiveness for key language understanding tasks .
Outcome: The proposed technique outperforms naive supervised training in mislabeled data and label expansion settings.
Do Pretrained Contextual Language Models Distinguish between Hebrew Homograph Analyses? (2023.eacl-main)

Copied to clipboard

Challenge: Semitic morphologically-rich languages are characterized by extreme word ambiguity . many of the words are homographs with multiple possible analyses .
Approach: They evaluate existing models for Hebrew homographs using word-piece embeddings . they find they are more effective when the number of word-part splits is limited .
Outcome: The proposed models outperform non-contextualized embeddings on Hebrew homograph challenge sets.
Fine-grained Category Discovery under Coarse-grained supervision with Hierarchical Weighted Self-contrastive Learning (2022.emnlp-main)

Copied to clipboard

Challenge: Existing methods for novel category discovery focus on the scenario where known and novel categories are of the same granularity.
Approach: They propose a novel scenario for fine-grained category discovery under coarse-grain supervision that allows for adapting models to categories of different granularity from known ones.
Outcome: The proposed model can adapt models to categories of different granularity from known ones and reduce labeling cost.
LMCap: Few-shot Multilingual Image Captioning by Retrieval Augmented Language Model Prompting (2023.findings-acl)

Copied to clipboard

Challenge: Existing captioning models are mostly available on English or are specialised for other high-resource languages.
Approach: They propose an image-blind few-shot multilingual captioning model that prompts a language model with retrieved captions.
Outcome: The proposed model is competitive with fully-supervised multilingual captioning models without requiring any supervised training on captioning data.
Disentangled Learning of Stance and Aspect Topics for Vaccine Attitude Detection in Social Media (2022.naacl-main)

Copied to clipboard

Challenge: Existing approaches to detect vaccine attitudes on social media require abundant annotations and pre-defined aspect categories.
Approach: They propose a semi-supervised approach to detect vaccine attitudes on social media . they use an autoencoding architecture to learn from unlabelled data the topical information of the domain .
Outcome: The proposed model outperforms existing aspect-based models on stance detection and tweet clustering.
Natural Language to Structured Query Generation via Meta-Learning (N18-2)

Copied to clipboard

Challenge: Conventional supervised training is a pervasive paradigm for NLP problems . however, examples of the same problem may vary widely . a few-shot meta-learning scenario is used to learn multiple models .
Approach: They propose a learning protocol that treats each example as a unique pseudo-task . they use a few-shot meta-learning scenario to reduce the original learning problem to a single example .
Outcome: The proposed learning protocol achieves 1.1%–5.4% accuracy gains over non-meta-learning counterparts on a WikiSQL dataset.
Read to Hear: A Zero-Shot Pronunciation Assessment Using Textual Descriptions and LLMs (2025.emnlp-main)

Copied to clipboard

Challenge: Automatic pronunciation assessment is typically performed by acoustic models trained on audio-score pairs.
Approach: They propose a zero-shot, textual description-based Pronunciation Assessment approach that utilizes human-readable representations of speech signals fed into an LLM to assess pronunciation accuracy and fluency.
Outcome: The proposed approach is cost-efficient and competitive in performance . it significantly improves the performance of conventional audio-score-trained models on out-of-domain data .
SHAPED: Shared-Private Encoder-Decoder for Text Style Adaptation (N18-1)

Copied to clipboard

Challenge: Experimentally, we find that the proposed models consistently outperform models that encapsulate single-style or average-style language generation capabilities.
Approach: They propose a family of model architectures capable of capturing both generic language characteristics via shared model parameters, as well as particular style characteristics via private model parameters.
Outcome: The proposed models outperform models that encapsulate single-style or average-style language generation capabilities.
Product Description and QA Assisted Self-Supervised Opinion Summarization (2024.findings-naacl)

Copied to clipboard

Challenge: Existing methods to generate opinion summarization without supervised training data are limited due to the lack of additional sources.
Approach: They propose a synthetic dataset creation strategy that leverages reviews and additional sources to generate a pseudo-summary.
Outcome: The proposed approach achieves 14.5% improvement in ROUGE-1 F1 over existing models.
Implicit Discourse Relation Classification For Nigerian Pidgin (2025.coling-main)

Copied to clipboard

Challenge: Existing discourse parsing tools are not available for Nigerian Pidgin (NP) this task requires supervised training and requires prompting.
Approach: They propose to use implicit discourse relation classification (IDRC) for Nigerian Pidgin, which requires supervised training.
Outcome: The proposed framework outperforms baseline and NP IDR classifiers in f1 scores.
Question Generation from SQL Queries Improves Neural Semantic Parsing (D18-1)

Copied to clipboard

Challenge: Using question generation, we learn a semantic parser with 30% of the supervised training data.
Approach: They propose to use question generation to learn a semantic parser with less supervised training data.
Outcome: The proposed method improves the state-of-the-art model with less training data.
Learning to Retrieve Passages without Supervision (2022.naacl-main)

Copied to clipboard

Challenge: Dense retrievers for open domain question answering have been shown to achieve impressive performance by training on large datasets of question-passage pairs.
Approach: They propose to use recurring spans to create pseudo examples for contrastive learning.
Outcome: The proposed model outperforms all pretrained baselines on a wide range of ODQA datasets and is competitive with BM25, a strong sparse baseline.
Recommendation as a Communication Game: Self-Supervised Bot-Play for Goal-oriented Dialogue (D19-1)

Copied to clipboard

Challenge: Traditional recommendation systems produce static rather than interactive recommendations invariant to a user’s specific requests, clarifications, or current mood.
Approach: They use a goal-driven recommendation dialogue dataset to develop an end-to-end dialogue system that can simultaneously converse and recommend.
Outcome: The proposed system can converse and recommend movies to humans without considering the task goal itself.
STAR: Constraint LoRA with Dynamic Active Learning for Data-Efficient Fine-Tuning of Large Language Models (2024.findings-acl)

Copied to clipboard

Challenge: Existing studies show that supervised training is still necessary for complex reasoning tasks.
Approach: They propose a method to integrate uncertainty-based active learning and LoRA to effectively integrate the two methods.
Outcome: The proposed approach outperforms baseline models on three reasoning tasks.
SMART: Self-Aware Agent for Tool Overuse Mitigation (2025.findings-acl)

Copied to clipboard

Challenge: Current Large Language Models (LLMs) lack self-awareness to balance reasoning and tool use, increasing computational overhead.
Approach: They propose a paradigm that enhances an agent’s self-awareness to optimize task handling and reduce tool overuse.
Outcome: The proposed model reduces tool use by 24% while improving performance by over 37%.
Learning to Denoise Distantly-Labeled Data for Entity Typing (N19-1)

Copied to clipboard

Challenge: Distantly-labeled data can be used to scale up statistical models, but it is noisy . specialized probabilistic models can be employed to scale the training of models, however, they require sophisticated probabilistic inference for the training.
Approach: They propose a method for denoising and denoising noisy data with supervised training.
Outcome: The proposed method outperforms models trained on clean and denoised data on an ultra-fine entity typing task.
G-Transformer for Document-Level Machine Translation (2021.acl-long)

Copied to clipboard

Challenge: Existing work extends translation unit from single sentence to multiple sentences.
Approach: They propose to introduce locality assumption as an inductive bias into Transformer and reduce the hypothesis space of attention from target to source.
Outcome: The proposed model achieves state-of-the-art BLEU scores on three benchmark datasets.
UCoder: Unsupervised Code Generation by Internal Probing of Large Language Models (2026.findings-acl)

Copied to clipboard

Challenge: Large language models (LLMs) have demonstrated remarkable capabilities in code generation tasks, but their effectiveness relies on supervised training with extensive labeled data and computational resources.
Approach: They propose an unsupervised method that leverages Internal Probing of Large language models for Code generation without any external corpus, even unlabeled code snippets.
Outcome: The proposed method can achieve competitive performance compared to supervised approaches while reducing the dependency on labeled data and computational resources.
Self-Supervised Behavior Cloned Transformers are Path Crawlers for Text Games (2023.findings-emnlp)

Copied to clipboard

Challenge: Text game agents are often modeled using reinforcement learning, but their performance is limited.
Approach: They propose a self-supervised behavior cloning transformer for text games . they explore trajectories that lead to reward within the games and then train small models . their approach consistently uncovers generalizable training data, achieving 90% performance of supervised systems across three benchmark text games.
Outcome: The proposed model achieves 90% performance on three text games.
Multitasking Framework for Unsupervised Simple Definition Generation (2022.acl-long)

Copied to clipboard

Challenge: Existing definition generation tasks require a dictionary with complex definitions and a corpus containing arbitrary simple texts to generate them.
Approach: They propose a multitasking framework SimpDefiner that only requires a standard dictionary with complex definitions and a corpus containing arbitrary simple texts.
Outcome: The proposed framework outperforms the baseline model by a 1.77 SARI score on the English dataset, and raises the proportion of the low level (HSK level 1-3) words in Chinese definitions by 3.87%.
Leveraging pre-trained language models for linguistic analysis: A case of argument structure constructions (2024.emnlp-main)

Copied to clipboard

Challenge: Argument structure constructions (ASCs) are lexicogrammatical patterns at the clausal level.
Approach: They evaluate the effectiveness of pre-trained language models in identifying argument structure constructions . they use supervised training with RoBERTa and prompt-guided annotation with GPT-4 .
Outcome: The proposed model outperforms the gold-standard model on three methods . the results show that the model performs better on gold-standardized data .
The student becomes the master: Outperforming GPT3 on Scientific Factual Error Correction (2023.findings-emnlp)

Copied to clipboard

Challenge: Existing methods for Factual Claim Correction rely on a verification model to guide the correction process.
Approach: They propose a claim correction system that does not require a verifier but outperforms existing methods by a considerable margin.
Outcome: The proposed system outperforms existing methods by a considerable margin on the SciFact dataset, 77% on SciFACT-Open and 72.75% on the CovidFact data set.
ConGen: Unsupervised Control and Generalization Distillation For Sentence Representation (2022.findings-emnlp)

Copied to clipboard

Challenge: Sentence representations are essential in many NLP tasks operating at the sentence level.
Approach: They propose an unsupervised sentence representation method to reduce the supervised-unsupervised performance gap for smaller models.
Outcome: The proposed method outperforms supervised training on STS, text classification, and natural language inference tasks on smaller models.
Perhaps PTLMs Should Go to School – A Task to Assess Open Book and Closed Book QA (2021.emnlp-main)

Copied to clipboard

Challenge: Taking the exam closed book, but having read the textbook, yields at best minor improvement (56%), suggesting that the PTLM may not have “understood” the textbook (or perhaps misundersttoo the questions).
Approach: They propose to use pre-trained language models to answer questions from introductory college textbooks and hundreds of true/false statements based on review questions written by the authors.
Outcome: The proposed task includes two college-level introductory texts in the social sciences (American Government 2e) and humanities (U.S. History).
Simple Entity-Centric Questions Challenge Dense Retrievers (2021.emnlp-main)

Copied to clipboard

Challenge: Open-domain question answering has exploded in popularity due to the success of dense retrieval models.
Approach: They construct a set of simple, entity-rich questions based on facts from Wikidata and test their models against supervised datasets.
Outcome: The proposed model outperforms sparse retrieval methods on open-domain question answering datasets by a large margin.
DiffChat: Learning to Chat with Text-to-Image Synthesis Models for Interactive Image Creation (2024.findings-acl)

Copied to clipboard

Challenge: a novel method to align Large Language Models to "chat" with prompt-as-input Text-to-Image Synthesis models is proposed . a user-specified instruction can be used to create a high quality image .
Approach: They propose a method to align Large Language Models to "chat" with prompt-as-input Text-to-Image Synthesis models for interactive image creation.
Outcome: The proposed method can exhibit superior performance than baseline models and strong competitors based on automatic and human evaluations.
ProGene - A Large-scale, High-Quality Protein-Gene Annotated Benchmark Corpus (2020.lrec-1)

Copied to clipboard

Challenge: Genes and proteins are fundamental entities of molecular genetics and are important for precision medicine.
Approach: They propose to use a corpus of gene and protein names to cope with this class of named entities in a large-scale annotation campaign at the Jena University Language & Information Engineering lab.
Outcome: The proposed corpus is an overall subdomain-independent corpus . it consists of 3,308 MEDLINE abstracts with over 36k sentences and more than 960k tokens annotated with nearly 60k named entity mentions.
Prompted Aspect Key Point Analysis for Quantitative Review Summarization (2024.acl-long)

Copied to clipboard

Challenge: Recent abstractive approaches generate KPs based on sentences, resulting in overlapping and hallucinated opinions.
Approach: They propose to use supervised learning to extract short sentences as key points before matching them to review comments for quantification of KP prevalence.
Outcome: The proposed framework achieves state-of-the-art performance on Yelp and SPACE.
Cross-lingual Spoken Language Understanding with Regularized Representation Alignment (2020.emnlp-main)

Copied to clipboard

Challenge: despite promising results, current cross-lingual models suffer from imperfect cross-linguistic representation alignments between the source and target languages, which makes the performance sub-optimal.
Approach: They propose a regularization approach to align word-level and sentence-level representations across languages without external resources.
Outcome: The proposed model outperforms state-of-the-art models in few-shot and zero-shot scenarios and achieves comparable performance to supervised training with all training data.
Unsupervised Dual Paraphrasing for Two-stage Semantic Parsing (2020.acl-main)

Copied to clipboard

Challenge: Existing semantic parsing frameworks rely on nontrivial human labor to generate canonical utterances.
Approach: They propose a framework that uses an unsupervised paraphrase model to parse canonical utterances.
Outcome: The proposed framework is effective and compatible with supervised training.
Lost in Literalism: How Supervised Training Shapes Translationese in LLMs (2025.acl-long)

Copied to clipboard

Challenge: Large language models exhibit translationese errors and generate unexpected unnatural translations . Neural machine translation (NMT) has become the dominant method in machine translation research .
Approach: They evaluate the prevalence of translationese in LLM-generated translations and investigate its roots during supervised fine-tuning.
Outcome: The proposed methods reduce translationese while improving translation naturalness . the proposed methods are validated by human evaluations and automatic metrics .
An automatic model and Gold Standard for translation alignment of Ancient Greek (2022.lrec-1)

Copied to clipboard

Challenge: Using a manual annotation tool, we evaluated the performance of various automatic translation alignment models for Ancient Greek.
Approach: They propose a fine-tuning strategy that employs unsupervised training with mono- and bilingual texts and supervised training using manually aligned sentences.
Outcome: The proposed model outperforms the standard on language pairs that were not part of the training data.
Analyze, Generate and Refine: Query Expansion with LLMs for Zero-Shot Open-Domain QA (2024.findings-acl)

Copied to clipboard

Challenge: Existing methods like GAR and EAR rely heavily on supervised training and struggle to maintain effectiveness across domains and datasets.
Approach: They propose a QE approach based on a three-step prompting strategy to enhance query expansion by broadening the scope of queries with additional relevant texts.
Outcome: The proposed approach outperforms state-of-the-art methods in out-domain zero-shot scenarios and outperformed existing methods in end-to-end evaluations.
Self-training Language Models for Arithmetic Reasoning (2024.findings-emnlp)

Copied to clipboard

Challenge: Recent work improves the reasoning capabilities of language models by scaling training data to more diverse or complex collections, but reaching further improvements becomes exceedingly expensive.
Approach: They propose to use implicit feedback to improve models' reasoning capabilities by training from implicit feedback.
Outcome: The proposed model can reach a correct result in +13.9% and +25.9% more cases than previous models, underlining the importance of actuality of self-training feedback.
Learning Opinion Summarizers by Selecting Informative Reviews (2021.emnlp-main)

Copied to clipboard

Challenge: supervised summarization has been traditionally approached with unsupervised, weakly-supervised and few-shot learning techniques.
Approach: They propose to combine a large dataset of opinion summaries with user reviews to form a supervised summarizer.
Outcome: The proposed method improves the quality of summarization and reduces hallucinations in the summarizer.
Learning to Generate Overlap Summaries through Noisy Synthetic Data (2022.emnlp-main)

Copied to clipboard

Challenge: Existing training data for seq-to-seq models is limited due to the lack of available training data.
Approach: They propose a data augmentation technique which allows to create large amount of synthetic data for training a seq-to-seq model.
Outcome: The proposed technique performs better than pre-trained models on news domains and is close to the existing methods on golden training data.
MSMO-ABSA: Multi-Scale and Multi-Objective Optimization for Cross-Lingual Aspect-Based Sentiment Analysis (2026.acl-long)

Copied to clipboard

Challenge: Aspect-based sentiment analysis (ABSA) has seen success with English texts, but real-world social media interactions often involve multiple languages.
Approach: They propose a framework for cross-lingual ABSA that incorporates code-switched bilingual sentences into the language discriminator and consistency training modules to enhance cross-linguistic alignment.
Outcome: The proposed framework achieves cross-lingual sentence-level and aspect-level alignment, aligning features of aspect terms in different contextual environments.
Towards Zero-shot Learning for End-to-end Cross-modal Translation Models (2023.findings-emnlp)

Copied to clipboard

Challenge: End-to-end zero-shot speech translation model is based on a zero-shot approach, but it is less competitive because of the limited amount of data available for multiple modalities.
Approach: They propose an end-to-end zero-shot speech translation model that connects two pre-trained uni-modality modules via word rotator’s distance.
Outcome: The proposed model performs better than or as well as those of the CTC-based models and can be trained in an end-to-end style to avoid error propagation.
Rule-Guided Extraction: A Hierarchical Rule Optimization Framework for Document-Level Event Argument Extraction (2025.findings-emnlp)

Copied to clipboard

Challenge: Document-level event argument extraction (EAE) is a critical task in natural language processing.
Approach: They propose an LLM-driven HiErarchical Rule Optimization framework that iteratively generates and selects optimal hierarchical rules.
Outcome: The proposed framework outperforms few-shot supervised methods and outperformed state-of-the-art prompting baselines.
Aspect-Oriented Summarization for Psychiatric Short-Term Readmission Prediction (2025.emnlp-main)

Copied to clipboard

Challenge: Recent advances in large language models have enabled the automated processing of lengthy documents even without supervised training on a task-specific dataset.
Approach: They propose a method for processing the summaries of long documents using different aspect-oriented prompts and integrate the information signals from these different prompts for supervised training of transformer models.
Outcome: The proposed method improves on a high-impact task predicting readmissions from a psychiatric discharge using real-world data from four hospitals.
Negative Matters: Multi-Granularity Hard-Negative Synthesis and Anchor-Token-Aware Pooling for Enhanced Text Embeddings (2025.acl-long)

Copied to clipboard

Challenge: Text embedding models are used for various natural language processing tasks such as sentiment analysis, text clustering, and content-based information retrieval.
Approach: They propose a synthesis framework that leverages large language models to generate diverse negative samples with varying levels of similarity with the query.
Outcome: The proposed framework achieves state-of-the-art performance surpassing existing synthesis strategies with synthetic data and when combined with public retrieval datasets.

What is GenGO?

GenGO is an NLP powered publication search system. It currenctly indexes 30k+ papers from ACL Anthology, and implements multi-aspect summarization, semantic search, and more!

Information

About
Limitations